Preventing recurrence of deterministic failures

ABSTRACT

Systems, methods, and computer program products to perform an operation comprising receiving, by a management console managing a server, a request to perform an operation on the server, determining that performance of the operation is associated with a deterministic failure in the server, and blocking performance of the operation by the management console, thereby preventing an occurrence of the deterministic failure in the server.

BACKGROUND

The present disclosure relates to computer software, and morespecifically, to computer software to prevent recurrence ofdeterministic failures.

Providers of computing services often need to ensure that downtime isminimized. Providers can typically overcome or avoid hardware failuresusing redundancy features, concurrent maintenance, and other techniques.On the other hand, software failures may severely reduce systemavailability. Often, providers attempt to survive software failures byrestarting applications on the same or different servers, or byrelocating the application (or its virtual machine) to another computenode in the computing environment. However, these techniques may notsuffice, as some types of software failures may persist when using byrestarting or relocating applications.

SUMMARY

Aspects disclosed herein include systems, methods, and computer programproducts to perform an operation comprising receiving, by a managementconsole managing a server, a request to perform an operation on theserver, determining that performance of the operation is associated witha deterministic failure in the server, and blocking performance of theoperation by the management console, thereby preventing an occurrence ofthe deterministic failure in the server.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates a system to prevent recurrence of deterministicfailures, according to one aspect.

FIG. 2 illustrates a method to prevent recurrence of deterministicfailures, according to one aspect.

FIG. 3 illustrates a method to define prohibited operations, accordingto one aspect.

FIG. 4 depicts a cloud computing node, according to one aspect.

FIG. 5 depicts a cloud computing environment, according to one aspect.

FIG. 6 depicts abstraction model layers, according to one aspect.

DETAILED DESCRIPTION

Embodiments presented herein provide techniques for preventing users orapplications from performing operations that are known to result infailures. When a failure is first encountered, a management console mayrecord each operation that led to the failure, and store the operationas a prohibited operation in a data store. When a subsequent request isissued to perform the operation, the management console may referencethe data store to determine that the operation is prohibited, and blockperformance of the operation.

To maintain a high availability of services in a computing cluster, aprovider may use a set of management applications. As used herein, theterm “management application” refers to any application that managesworkloads, services, applications, and/or logical partitions (or virtualmachines). For example, management applications may include, withoutlimitation, applications to remotely restart, migrate, or clone servicesand/or logical partitions in the cluster. Specific examples ofmanagement applications include, without limitation, VMcontrol, WorkloadDeployment Manager, Smart Cloud Entry, Systems Director, and PowerVC byInternational Business Machines®.

In many cases, the management applications interact with a managementconsole to perform their intended operations. Generally, a managementconsole refers to an application that provides an interface forconfiguring and operating partitioned (or virtualized) systems, as wellas managing the physical and virtual resources of the systems. Oneexample of a management console is the Hardware Management Console byInternational Business Machines®. The management console may allowsystem administrators (or the management applications) to manage thesoftware configuration and operation of partitions in a server system,as well as to monitor and identify problems. For example, the managementconsole may be used to configure and manage logical partition profiles,perform dynamic logical partitioning (DLPAR) operations, and activateand manage capacity on demand resources.

When a user, the management applications, or the management consoleperform operations, these operations may lead to hardware and/orsoftware failures, reducing the availability of services provided by thecomputing cluster. Generally, software failures may be categorized intotwo types, namely deterministic failures and non-deterministic failures.As used herein, a “deterministic failure” refers to failures that mayconsistently be exposed by following specific paths, steps, or codeflow, such that the failure can be easily recreated. A“non-deterministic” failure refers to failures which can happen at anytime, and are not easily recreated, such as memory corruption,miscompares, and timing/serialization issues.

Non-deterministic failures are often remedied by restarting services orrelocating services to different servers. However, deterministicfailures may recur with these techniques, as users or applications maycontinue to perform the same actions without knowledge that theseactions caused the deterministic failure. As such, the managementconsole may identify the offending actions that cause deterministicfailures. The management console may maintain a data store includingeach prohibited (or otherwise unsupported) operation. The managementconsole may then prohibit those operations from being subsequentlyperformed, regardless of the requesting party (such as the user, amanagement application, or the management console itself).

Example system management operations include, without limitation, DLPARoperations, hibernation operations, relocation operations, and the like.For example, a user may initiate a DLPAR operation on a logicalpartition, which may cause the logical partition to crash due to adefect in the memory management logic of the logical partition. When thefailure occurs, the management console may identify the steps thatcaused the failure (in this case, invocation of the memory managementlogic). An indication that the memory management logic is flawed maythen be stored by the management console. When subsequent calls are madeto the memory management logic, the management console may prohibit thecall from being performed. Sometimes, these offending calls may be madeinadvertently by a user, automatically by the management console, or bysome other operation (such as page migration, affinity changes, and thelike). Regardless of the entity making the call to the flawed memorymanagement logic, the management console will prevent the call frombeing made, and therefore prevent the memory management logic fromcausing a recurrence of the same failure.

FIG. 1 illustrates a system 100 to prevent recurrence of deterministicfailures, according to one aspect. The networked system 100 includes acomputer 102 connected to other computers via a network 130. In general,the network 130 may be a telecommunications network and/or a wide areanetwork (WAN). In a particular embodiment, the network 130 is theInternet. In at least one aspect, the system 100 is a cloud computingcluster where servers 160 host logical partitions 161 (or virtualmachines) that provide services (not pictured) to clients.

The computer 102 generally includes a processor 104 which obtainsinstructions and data via a bus 120 from a memory 106 and/or a storage108. The computer 102 may also include one or more network interfacedevices 118, input devices 122, and display devices 124 connected to thebus 120. The computer 102 is generally under the control of an operatingsystem (not shown). Examples of operating systems include the UNIX®operating system, versions of the Microsoft® Windows® operating system,and distributions of the Linux® operating system. (UNIX® is a registeredtrademark of The Open Group in the United States and other countries.Microsoft® and Windows® are trademarks of Microsoft Corporation in theUnited States, other countries, or both. Linux® is a registeredtrademark of Linus Torvalds in the United States, other countries, orboth.) More generally, any operating system supporting the functionsdisclosed herein may be used. The processor 104 is a programmable logicdevice that performs instruction, logic, and mathematical processing,and may be representative of one or more CPUs. The network interfacedevice 118 may be any type of network communications device allowing thecomputer 102 to communicate with other computers via the network 130.

The storage 108 is representative of hard-disk drives, solid statedrives, flash memory devices, optical media and the like. Generally, thestorage 108 stores application programs and data for use by the computer102. In addition, the memory 106 and the storage 108 may be consideredto include memory physically located elsewhere; for example, on anothercomputer coupled to the computer 102 via the bus 120.

The input device 122 may be any device for providing input to thecomputer 102. For example, a keyboard and/or a mouse may be used. Theinput device 122 represents a wide variety of input devices, includingkeyboards, mice, controllers, and so on. Furthermore, the input device122 may include a set of buttons, switches or other physical devicemechanisms for controlling the computer 102. The display device 124 mayinclude output devices such as monitors, touch screen displays, and soon.

As shown, the memory 106 contains the management console 112, which isan application that provides an interface for configuring and operatingpartitioned (or virtualized) systems, such as the logical partitions 161on the servers 160. The management console 112 is further configured toprevent prohibited actions or operations from being performed when thoseactions or operations have previously been observed to causedeterministic failures in the servers 160 and/or the logical partitions161. When a deterministic failure is first encountered, the managementconsole 112 may identify, in the logs 121, steps or actions taken thatled to the deterministic failure. The management console 112 may thenstore an indication of the identified actions in the prohibitedoperations 120. In at least some aspects, the management console 112 maypresent the operations identified in the logs 121 to a user, who maythen refine or edit the operations in order to create an entry in theprohibited operations 120. In addition, a user may manually define oneor more operations as being prohibited in the prohibited operations 120without first encountering a deterministic failure.

When a user or management application 114 subsequently requests toperform an operation, the management console 112 may reference theprohibited operations 120 in order to determine whether the requestedoperation is associated with a deterministic failure. If the requestedoperation is associated with a deterministic failure, the managementconsole 112 does not permit performance of the requested operation,preventing the failure. The management console 112 may notify a userrequesting the operation that the operation is blocked and will not beperformed.

In some aspects, the system 100 may include multiple computers 102executing instances of the management console 112, such as in the caseof a cloud computing environment or computing cluster. In such cases, afirst management console 112, when storing an indication of a prohibitedoperation in the prohibited operations 120, may share the indicationwith other instances of the management console 112 executing in thesystem. Doing so allows the management consoles 112 to share informationrelated to deterministic failures, and prevent those operations frombeing performed anywhere in the system 100.

In many cases, a fix for operations associated with deterministicfailures may be created. In such a case, if the fix remedies the issuesuch that the operation is no longer associated with a deterministicfailure, a user or the management console 112 may update the prohibitedoperations 120 such that the operation is no longer prohibited. Themanagement console 112 may update the prohibited operations 120 inresponse to installation of the fix. Once the prohibited operation isfixed, the management console 112 does not block the operation frombeing performed. For example, the previously discussed flawed memorymanagement logic may be updated so that the memory management logic nowfunctions properly. When the fix is made to the memory management logic,the management console 112 (or a user) may update prohibited operations120 such that the flawed memory management logic is no longer associatedwith a deterministic failure. When a subsequent call is made to thememory management logic, the management console allows the memorymanagement logic to perform its requested function. Generally, even inthe absence of installing a fix, a user may remove operations from theprohibited operations 120, allowing the removed operations to besubsequently performed.

In addition to operations associated with failures, the managementconsole 112 may also prevent unsupported operations from beingperformed. For example, if a user requests to add eight processors to alogical partition 161, but the customer's service level agreement onlyprovides up to four processors, the management console 112 may block therequest to add eight processors to the logical partition. Similarly, ifan operation is not supported by a particular deployment, the managementconsole 112 may block that operation. Doing so allows users to avoid theneed to reference lists of unsupported features and/or operationscurrently provided in system documentation.

The management applications 114 are systems management applications thatmanage workloads, services, applications, and/or logical partitions (orvirtual machines). In many cases, the operations performed by themanagement applications 114 are executed by automated scripts or webinterfaces to the management console 112. The management applications114 may perform activities such as load balancing using DLPAR, deployingnew logical partitions 161, configuration changes, optimization throughaffinity changes, relocation of logical partitions 161 on the servers160, and the like. However, these operations may lead to deterministicfailures. When identified, the management console 112 may identify theseoperations in the prohibited operations.

As shown, the storage 108 includes a reference to prohibited operations120 and the logs 121. The prohibited operations 120 indicate whatoperations are known to cause deterministic failures in the servers 160and/or logical partitions 161 (and/or any services hosted thereon). Theoperations may include any types of operations that may result in afailure, and if repeated, will recreate the same failure (generallyreferred to as deterministic failures). One example of such operationsare systems management operations, which may comprise operationsperformed by the management console 112 and/or the managementapplications 114. The prohibited operations 120 may also specifyunsupported operations in the servers 160 and/or logical partitions 161.When the management console 112 receives a request to perform anoperation, the management console 112 may first reference the prohibitedoperations 120. If the management console 112 determines a requestedoperation is prohibited and/or unsupported, the management console 112blocks performance of that operation. In one aspect, data in theprohibited operations 120 includes, the type of failure triggered by theoperation, under what conditions the failure is triggered, and how thefailure can be avoided. In addition, the prohibited operations 120 mayspecify a class of the specified failure. Examples of classes of failureinclude transient failures, permanent failures, recoverable failures,unrecoverable failures, and cosmetic failures. A transient failure maybe a failure that occurs only for certain input values while invoking afunction of the system. A permanent failure may occur for all inputvalues while invoking a function of the system. A recoverable failure isa failure that, upon occurrence, the system recovers with or withoutoperator intervention. An unrecoverable failure is a failure thatrequires a restart of the system in order to be corrected. A cosmeticfailure is a class of failure that causes minor errors, and does notlead to incorrect results.

The management console 112 may tailor an action taken for a givenoperation based on the associated class of failure. For example, themanagement console 112 may warn a user that an operation requested isexpected to result in a cosmetic failure. Because the cosmetic failuredoes not lead to incorrect results, the user may approve the operation.However, for other classes of errors, the management console 112 doesnot allow the user to override the block on performing the operation.

The logs 121 stores a list of the operations performed by the managementconsole 112 and/or the management applications 114. In at least oneaspect, the management console 112 writes each operation it performs (orallows to be performed) to the logs 121. By maintaining a list ofoperations in the logs 121, the management console 112 identifies themost recent operations that lead to failures. In such cases, themanagement console 112 may store the operations in the prohibitedoperations 120, such that the management console 112 can prevent theerrors from recurring by blocking requests to perform those operations.

FIG. 2 illustrates a method 200 to prevent recurrence of deterministicfailures, according to one aspect. Generally, the steps of method 200prohibit operations from being performed if those operations known tocause a deterministic failure. For example, in a cloud computingenvironment, the management console 112 may constantly migrateresources, deploy new resources, and the like, all of which occurstransparently to a user. If a software bug is associated with migratinga logical partition that can potentially disable a target node, themanagement console 112 may trigger this error with each migration of thelogical partition. In such a scenario, it s not practical to shut downthe entire cloud operation just to contain a failure caused by aspecific activity associated with the logical partition. Instead, themanagement console 112 may block the activity associated with thelogical failure, in this case migrating the offending logical partitionsto new servers or nodes.

At step 210, the management console 112 (or a user) may defineprohibited operations. Prohibited operations generally correspond tooperations identified as causing deterministic failures, or as not beingsupported. The operations are stored in the prohibited operations 120.At step 220, the management console 112 may provide a user interface todefine, modify, and/or remove operations from the prohibited operations120. For example, the management console 112 may allow a user to removean entry in the prohibited operations 120 corresponding to an operationthat has been fixed by a software update. At step 230, the managementconsole 112 may receive a request to perform an operation. The operationmay be any type of operation, such as a DLPAR operation, hibernationoperation, relocation operation, and the like.

At step 240, the management console 112 may determine whether theoperation is prohibited by referencing the prohibited operations 120. Atstep 250, the management console 112 may perform the operation upondetermining the operation is not prohibited, or was previouslyprohibited but the failure the operation caused has now been addressedby a software update. At block 260, the management console 112 blocksperformance of the requested operation upon determining that theoperation is defined as a prohibited operation in the prohibitedoperations 120. At block 270, the management console 112 may output anindication to the user that the prohibited operation will not beperformed.

FIG. 3 illustrates a method 300 corresponding to step 210 to defineprohibited operations, according to one aspect. Generally, the steps ofthe method 300 identify prohibited operations that trigger deterministicfailures on a server hosting one or more logical partitions. The methodbegins at step 310, where the management console 112 receives a requestto perform an operation. At step 320, the management console 112performs a loop including steps 330-370 for each step of the requestedoperation. At step 330, the management console 112 performs the currentstep of the requested operation. At step 340, the management console 112determines whether a deterministic failure has occurred. In at least oneaspect, the management console 112 may determine that a deterministicfailure has occurred by monitoring an “activity/code path” once thefailure is determined for the first time. Generally, if a set ofoperations (or activities), when performed, result in a failure havingthe same failure profile (which may include an error code, type oferror, and the like), the management console 112 can determine that theoperations result in a deterministic failure.

For example, if a user attempts to add an operation to add 10 GB ofmemory to a machine, the user may follow specific steps to add thememory to the machine. If the operation results in a failure, themanagement console 112 may trace back the list of steps followed by theuser (from the logs 120) before the failure, and save the steps in theprohibited operations 121 as a “suspected operation” along with afailure signature, which may include metadata such as an error code,type of error, and the like. Whenever this suspected operation isperformed again (i.e., adding 10 GB of memory), if the same failure withthe same failure signature (error code, type of error, etc.) isencountered, the management console 112 will determine that the failureis a deterministic failure, and update the prohibited operations 121 toreflect that the suspected operation is now a prohibited operation.Returning to step 340, if a failure has not occurred, the managementconsole 112 proceeds to step 370, where the management console 112determines whether more steps in the operation remain. If more stepsremain, the management console 112 returns to step 320. If no more stepsremain in the operation, the method 300 ends.

Returning to step 340, if a deterministic failure has occurred, themanagement console 112 identifies the steps taken that caused thefailure from the logs 120 (step 350). At step 360, the managementconsole 112 may save the steps identified from the logs 120 as anoperation in the prohibited operations 120 that cannot be performeduntil the operation no longer causes the deterministic failure. Method300 concludes once the management console 112 saves the operation to theprohibited operations 120.

Advantageously, aspects disclosed herein prevent repeat occurrences ofdeterministic failures in a computing cluster. The management console112 may identify operations that trigger deterministic failures, andstore these operations as prohibited operations that cannot be repeateduntil a fix is applied (or the operations are otherwise removed from theprohibited operations 120). When receiving a request to perform anoperation, the management console 112 may first determine whether theoperation is defined as a prohibited operation. If the operation hasbeen prohibited, the management console 112 will block performance ofthe operation. In some aspects, the management console 112 may alsoinform a user that performance of the operation has been blocked.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

For convenience, the Detailed Description includes the followingdefinitions which have been derived from the “Draft NIST WorkingDefinition of Cloud Computing” by Peter Mell and Tim Grance, dated Oct.7, 2009, which is cited in an IDS filed herewith, and a copy of which isattached thereto.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 4, a schematic of an example of a cloud computingnode is shown. Cloud computing node 10 is only one example of a suitablecloud computing node and is not intended to suggest any limitation as tothe scope of use or functionality of embodiments of the inventiondescribed herein. Regardless, cloud computing node 10 is capable ofbeing implemented and/or performing any of the functionality set forthhereinabove.

In cloud computing node 10 there is a computer system/server 12, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 12 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

As shown in FIG. 4, computer system/server 12 in cloud computing node 10is shown in the form of a general-purpose computing device. Thecomponents of computer system/server 12 may include, but are not limitedto, one or more processors or processing units 16, a system memory 28,and a bus 18 that couples various system components including systemmemory 28 to processor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via I/O interfaces22. Still yet, computer system/server 12 can communicate with one ormore networks such as a local area network (LAN), a general wide areanetwork (WAN), and/or a public network (e.g., the Internet) via networkadapter 20. As depicted, network adapter 20 communicates with the othercomponents of computer system/server 12 via bus 18. It should beunderstood that although not shown, other hardware and/or softwarecomponents could be used in conjunction with computer system/server 12.Examples, include, but are not limited to: microcode, device drivers,redundant processing units, external disk drive arrays, RAID systems,tape drives, and data archival storage systems, etc.

Referring now to FIG. 5, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 5 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 6, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 5) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 6 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include mainframes, in oneexample IBM® zSeries® systems; RISC (Reduced Instruction Set Computer)architecture based servers, in one example IBM pSeries® systems; IBMxSeries® systems; IBM BladeCenter® systems; storage devices; networksand networking components. Examples of software components includenetwork application server software, in one example IBM WebSphere®application server software; and database software, in one example IBMDB2® database software. (IBM, zSeries, pSeries, xSeries, BladeCenter,WebSphere, and DB2 are trademarks of International Business MachinesCorporation registered in many jurisdictions worldwide)

Virtualization layer 62 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers;virtual storage; virtual networks, including virtual private networks;virtual applications and operating systems; and virtual clients.

In one example, management layer 64 may provide the functions describedbelow. Resource provisioning provides dynamic procurement of computingresources and other resources that are utilized to perform tasks withinthe cloud computing environment. Metering and Pricing provide costtracking as resources are utilized within the cloud computingenvironment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.Prohibited operations prevent users or applications from performingoperations that may result in a deterministic failure, as describedherein. Service level management provides cloud computing resourceallocation and management such that required service levels are met.Service Level Agreement (SLA) planning and fulfillment providepre-arrangement for, and procurement of, cloud computing resources forwhich a future requirement is anticipated in accordance with an SLA.

Workloads layer 66 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation; software development and lifecycle management; virtualclassroom education delivery; data analytics processing; user portal;and transaction processing.

The descriptions of the various aspects have been presented for purposesof illustration, but are not intended to be exhaustive or limited to theembodiments disclosed. Many modifications and variations will beapparent to those of ordinary skill in the art without departing fromthe scope and spirit of the described embodiments. The terminology usedherein was chosen to best explain the principles of the embodiments, thepractical application or technical improvement over technologies foundin the marketplace, or to enable others of ordinary skill in the art tounderstand the embodiments disclosed herein.

As will be appreciated by one skilled in the art, aspects may beembodied as a system, method or computer program product. Accordingly,aspects may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “circuit,”“module” or “system.” Furthermore, aspects may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects are described herein with reference to flowchart illustrationsand/or block diagrams of methods, apparatus (systems) and computerprogram products. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

What is claimed is:
 1. A method, comprising: receiving, by a managementconsole managing a server, a request to perform an operation on theserver; determining that performance of the operation is associated witha deterministic failure in the server; and blocking performance of theoperation by the management console and by operation of one or morecomputer processors, thereby preventing an occurrence of thedeterministic failure in the server.
 2. The method of claim 1, whereinthe operation is a system management operation, further comprising:prior to receiving the request: receiving an earlier request to performthe system management operation on the server; performing the systemmanagement operation on the server; determining that performing thesystem management operation on the server causes the deterministicfailure in the server; and storing, by the management console, anindication that the system management operation is a prohibitedoperation.
 3. The method of claim 2, further comprising: sharing, by themanagement console managing the server, the indication with a secondmanagement console, wherein the second management console manages asecond server.
 4. The method of claim 1, further comprising: outputting,by the management console, an indication that performance of theoperation has been blocked.
 5. The method of claim 1, whereindetermining that the operation is associated with the deterministicfailure comprises: identifying, in a data store storing a set ofprohibited operations, an entry associated with the operation.
 6. Themethod of claim 5, further comprising: upon determining that performanceof the operation no longer results in the deterministic failure,removing the entry associated with the operation from the data store. 7.The method of claim 1, wherein the operation is a system managementoperation, wherein the deterministic failure comprises a failure thatoccurs each time a set of steps of the system management operation areperformed, wherein the server is part of a computing cluster comprisinga plurality of servers, wherein a plurality of management consolesmanage a respective subset of the plurality of servers in the cluster,wherein the request is issued by at least one of: (i) a user, and (ii) amanagement application configured to provide high availability of theserver in the cluster.
 8. A system, comprising: a server; and amanagement console configured to manage the server, wherein themanagement console is configured to perform an operation comprising:receiving, by a management console managing a server, a request toperform an operation on the server; determining that performance of theoperation is associated with a deterministic failure in the server; andblocking performance of the operation by the management console, therebypreventing an occurrence of the deterministic failure in the server. 9.The system of claim 8, wherein the operation is a system managementoperation, wherein the operation further comprises: prior to receivingthe request: receiving an earlier request to perform the systemmanagement operation on the server; performing the system managementoperation on the server; determining that performing the systemmanagement operation on the server causes the deterministic failure inthe server; and storing, by the management console, an indication thatthe system management operation is a prohibited operation.
 10. Thesystem of claim 9, the operation further comprising: sharing, by themanagement console managing the server, the indication with a secondmanagement console, wherein the second management console manages asecond server.
 11. The system of claim 8, the operation furthercomprising: outputting, by the management console, an indication thatperformance of the operation has been blocked.
 12. The system of claim8, wherein determining that the operation is associated with thedeterministic failure comprises: identifying, in a data store storing aset of prohibited operations, an entry associated with the operation.13. The system of claim 12, the operation further comprising: upondetermining that performance of the system management operation nolonger results in the deterministic failure, removing the entryassociated with the operation from the data store.
 14. The system ofclaim 8, wherein the operation is a system management operation, whereinthe deterministic failure comprises a failure that occurs each time aset of steps of the system management operation are performed, whereinthe server is part of a computing cluster comprising a plurality ofservers, wherein a plurality of management consoles manage a respectivesubset of the plurality of servers in the cluster, wherein the requestis issued by at least one of: (i) a user, and (ii) a managementapplication configured to provide high availability of the server in thecluster.
 15. A computer program product, comprising: a computer-readablestorage medium having computer-readable program code embodied therewith,the computer-readable program code comprising: computer-readable programcode configured to receive, by a management console managing a server, arequest to perform an operation on the server; computer-readable programcode configured to determine that performance of the operation isassociated with a deterministic failure in the server; andcomputer-readable program code configured to block performance of theoperation by the management console, thereby preventing an occurrence ofthe deterministic failure in the server.
 16. The computer programproduct of claim 15, wherein the operation is a system managementoperation, further comprising: computer-readable program code configuredto, prior to receiving the request: receive an earlier request toperform the system management operation on the server; perform thesystem management operation on the server; determine that performing thesystem management operation on the server causes the deterministicfailure in the server; and store, by the management console, anindication that the system management operation is a prohibitedoperation.
 17. The computer program product of claim 16, furthercomprising: computer-readable program code configured to share, by themanagement console managing the server, the indication with a secondmanagement console, wherein the second management console manages asecond server.
 18. The computer program product of claim 15, furthercomprising: computer-readable program code configured to output, by themanagement console, an indication that performance of the operation hasbeen blocked.
 19. The computer program product of claim 15, whereindetermining that the system management operation is associated with thedeterministic failure comprises: identifying, in a data store storing aset of prohibited operations, an entry associated with the operation.20. The computer program product of claim 19, further comprising:computer-readable program code configured to, upon determining thatperformance of the system management operation no longer results in thedeterministic failure, remove the entry associated with the operationfrom the data store.